feat(init): add init command for guided Sentry project setup#283
Open
feat(init): add init command for guided Sentry project setup#283
Conversation
Adds `sentry init` wizard that walks users through project setup via the Mastra API, handling DSN configuration, SDK installation prompts, and local file operations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sends tags and metadata (CLI version, OS, arch, node version) with startAsync and resumeAsync calls so workflow runs are visible and filterable in Mastra Studio. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Import randomBytes and generate a hex trace ID so all suspend/resume calls within a single wizard run share one trace. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a synthetic parentSpanId to tracingOptions so all workflow run spans become siblings under the same parent instead of nesting by timestamp containment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The parentSpanId was creating artificial nesting - let the workflow engine handle span hierarchy naturally. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Display the branded SENTRY ASCII banner before the intro line for visual consistency with `sentry --help`. Make the "errors" feature always enabled in the feature multi-select so users cannot deselect error monitoring. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…pt, and source maps hint Route success-with-exitCode results to formatError so the --force hint is shown when Sentry is already installed. Fold the "Error Monitoring is always included" note into the multiselect prompt. Use a more approachable Source Maps hint. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Show a non-blocking info note about AI usage with a docs link before the first network call, and a review reminder before the success outro. Extract SENTRY_DOCS_URL constant to share between wizard-runner and clack-utils cancel message. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add @anthropic-ai/sdk and openai as devDependencies for the LLM-as-judge eval framework. Add opencode-lore dependency. Exclude test/init-eval/templates from biome linting since they are fixture apps, not source code. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add LLM-as-judge eval tests for the init wizard across all five platforms (Express, Next.js, Flask, React+Vite, SvelteKit). Each test runs the wizard end-to-end and asserts on SDK installation, Sentry.init presence, build success, and documentation accuracy via an LLM judge. Includes template apps, helper utilities (assertions, doc-fetcher, judge, platform configs), and feature-docs.json mapping. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a separate workflow for running init-eval tests on demand. Supports running a single platform or all platforms via matrix. Uses the init-eval GitHub environment for MASTRA_API_URL and OPENAI_API_KEY secrets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Store python-fastapi doc URLs as base paths (with trailing slash) like other platforms, and convert to .md at fetch time. This mirrors the pattern in cli-init-api and lets us return clean markdown directly instead of stripping HTML tags. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Sentry doc URLs for python-flask (getting-started, errors, tracing, logs, profiling) and add the shared python/profiling page to both flask and fastapi profiling entries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Sentry doc URLs for all nextjs features: getting-started, errors, logs, tracing, session replay, metrics, and profiling (browser + node). Sourcemaps left empty for now. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Sentry doc URLs for sveltekit features and add missing logs, metrics, and profiling features to the platform entry. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Sentry doc URLs for react-vite features and add missing logs, metrics, and profiling features to the platform entry. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Flask eval was using bare `pip install` which fails when pip isn't on PATH. Use the same venv pattern as fastapi. Also remove accidental opencode-lore runtime dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Semver Impact of This PR🟡 Minor (new features) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨
Bug Fixes 🐛Api
Formatters
Other
Internal Changes 🔧
🤖 This preview updates automatically when you update the PR. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
Codecov Results 📊✅ 45 passed | Total: 45 | Pass Rate: 100% | Execution Time: 0ms 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 94.97%. Project has 3854 uncovered lines. Files with missing lines (4)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
- Coverage 80.14% 78.89% -1.25%
==========================================
Files 120 127 +7
Lines 16316 18254 +1938
Branches 0 0 —
==========================================
+ Hits 13075 14400 +1325
- Misses 3241 3854 +613
- Partials 0 0 —Generated by Codecov Action |
Restrict GITHUB_TOKEN to contents:read as flagged by CodeQL. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update SvelteKit template with working deps (adapter-node, latest svelte/vite) and add required src files (app.d.ts, app.html). Use python3 instead of python for venv creation in Flask/FastAPI platforms. Add --concurrency 6 to init-eval test runner. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add push/pull_request triggers so the eval runs automatically alongside other CI checks. Keep workflow_dispatch for manual single-platform runs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move hardcoded numeric values, string literals, and exit codes into constants.ts for better readability and maintainability across the init module. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move regex to top-level constant (useTopLevelRegex) - Remove unused template literal (noUnusedTemplateLiteral) - Replace explicit `return undefined` with bare `return` (noUselessUndefined) - Apply formatter to both source and test files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add tests for local-ops (FS operations, command execution, patchset application), formatters (result/error display), help (banner/custom help output), interactive prompts (select/multiselect/confirm), and wizard-runner (TTY check, success/error paths, suspend/resume loop). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Keep both openai and marked dependencies, add test:init-eval script back, and take main's version (0.14.0-dev.0) and restructured package.json layout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…into feat/init-command
…coverage Include test/isolated in the test:unit coverage run so that existing comprehensive tests for wizard-runner and interactive modules count toward patch coverage. Add new tests for init command parsing, clack-utils utilities, cancel paths, and wizard-runner edge cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bun's mock.module() leaks between test files in the same run. Keep test:unit and test:isolated as separate invocations, add coverage flags to test:isolated, and merge lcov reports before upload. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
The init-command test's mock.module() for interactive.js was poisoning init-interactive.test.ts in the same bun test run. Moved to test/commands/ with a single wizard-runner.js mock instead of 7 redundant mocks — no other test in test/commands/ depends on that module. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bun's mock.module() leaks across files when run in a single process. Run each test/isolated/*.test.ts file in its own bun test invocation to ensure true mock isolation, accumulating LCOV coverage for CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Running each isolated test in its own bun process produces overlapping coverage for shared source files. Concatenating these LCOV files created duplicate SF entries that codecov counted as separate files, inflating line counts and dropping project coverage from 80% to 51%. Add script/merge-lcov.sh (awk) to deduplicate by source file, taking the max hit count per line, so codecov sees 51 unique files instead of 133 duplicate entries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Convert init-interactive and init-wizard-runner tests from isolated mock.module() pattern to spyOn() on namespace imports, eliminating mock leakage without process isolation. Also fix CI coverage merge to deduplicate LCOV entries via merge-lcov.sh. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
spyOn on local TS module exports doesn't intercept in bun on Linux, so wizard-runner must stay isolated with mock.module(). The interactive test remains in test/lib/init/ since it only spies on @clack/prompts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wrap fd operations in try/finally so fs.closeSync is always called, even if fs.readSync throws an I/O error. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rm shell
Add >, <, and & to SHELL_METACHARACTER_PATTERNS to prevent redirection
(e.g. `npm install foo > /arbitrary/path`) and background execution
(e.g. `npm install foo & curl evil.com`) in validated commands.
Replace hardcoded `spawn("sh", ...)` with `spawn(command, [], { shell: true })`
so Node selects the platform-appropriate shell (cmd.exe on Windows).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The remote workflow controls payload.cwd, but handleLocalOp never checked that it falls within options.directory. A misbehaving workflow could set cwd:"/" to escape the path sandbox entirely. Now cwd is validated at the top of handleLocalOp before any operation runs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
runWizard swallows errors and returns void, but most error paths were missing process.exitCode = 1, causing failed initializations to exit with code 0 in CI/CD pipelines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move createRun() into try/catch so network failures get graceful "Connection failed" message instead of an unhandled stack trace - Block shell expansion characters ($, ', ", \) in validateCommand to prevent bypass via ANSI-C quoting, variable expansion, and escapes - Remove unused stdout/stderr/stdin fields from WizardOptions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment on lines
+177
to
+183
| const payload = extractSuspendPayload(result, stepId); | ||
| if (!payload) { | ||
| spin.stop("Error", 1); | ||
| log.error(`No suspend payload found for step "${stepId}"`); | ||
| cancel("Setup failed"); | ||
| return; | ||
| } |
There was a problem hiding this comment.
Bug: A specific error path in the setup wizard exits with code 0 (success) instead of 1 (failure) when a suspend payload is missing, causing silent failures.
Severity: MEDIUM
Suggested Fix
Before the return; statement on line 182 in the if (!payload) block, add process.exitCode = 1; to ensure the process exits with a failure code, consistent with other error-handling paths in the function.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: src/lib/init/wizard-runner.ts#L177-L183
Potential issue: In the `runWizard` function, if `extractSuspendPayload` returns an
undefined payload, the function logs an error and then executes an early `return` on
line 182. This specific error path fails to set `process.exitCode = 1` before exiting.
As a result, a failed setup wizard run will incorrectly report a success status (exit
code 0) to the calling process. This behavior is inconsistent with all other failure
paths within the same function, which correctly set the exit code to 1, and can cause
CI/CD pipelines or automation scripts to misinterpret a failure as a success.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| { pattern: ">", label: "redirection (>)" }, | ||
| { pattern: "<", label: "redirection (<)" }, | ||
| { pattern: "&", label: "background execution (&)" }, | ||
| ]; |
There was a problem hiding this comment.
Parentheses bypass shell metacharacter and executable blocklist
High Severity
SHELL_METACHARACTER_PATTERNS blocks $( but not standalone ( or ). Since runSingleCommand uses shell: true, a command like (rm -rf .) passes all validation: no blocked metacharacters match, and the first-token extraction yields (rm whose path.basename is "(rm" — not in BLOCKED_EXECUTABLES. The shell then interprets (...) as a subshell, executing the blocked executable. This defeats both the metacharacter check and the executable blocklist for any command wrapped in parentheses.
Additional Locations (1)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
sentry init— an AI-powered wizard that walks users through adding Sentry to their project. It detects the platform, installs the SDK, instruments the code, and configures error monitoring, tracing, and session replay.Changes
initcommand backed by a Mastra AI workflow (hosted at getsentry/cli-init-api) that handles platform detection, SDK installation, and code instrumentationconstants.ts)test/init-eval/) — see belowEval suite
The eval suite validates that the wizard produces correct, buildable Sentry instrumentation for each supported platform. It uses a 3-phase test architecture:
Phase 1: Wizard run
Each test scaffolds a fresh project from a platform template, then runs the full
sentry initwizard against it. The wizard output (exit code, stdout/stderr, git diff, new files) is captured for the next phases.Phase 2: Hard assertions (deterministic)
Five code-based pass/fail checks that run without any LLM:
package.json/requirements.txt)Sentry.init(orsentry_sdk.init) appears in changed or new files___PUBLIC_DSN___,YOUR_DSN_HERE, etc.)npm run build/ equivalent passes after the wizard's changesPhase 3: LLM judge (per-feature)
For each feature (errors, tracing, replay, logs, profiling, etc.), an LLM judge scores correctness:
feature-docs.json)Platforms
6 platform templates are covered:
express/@sentry/nodenextjs/@sentry/nextjssveltekit/@sentry/sveltekitreact-vite/@sentry/reactpython-flask/sentry-sdkpython-fastapi/sentry-sdkRunning
bun run test:init-eval # all platformsRequires
SENTRY_AUTH_TOKEN,SENTRY_ORG,SENTRY_PROJECT, and optionallyOPENAI_API_KEY(LLM judge is skipped without it).Test Plan
bun run test:init-evalpasses for all 6 platformsbun run lintandbun run typecheckpass🤖 Generated with Claude Code